We present Hybrid Infused Reranking for Passages Retrieval (HYRR), a framework for training rerankers based on a hybrid of BM25 and neural retrieval models. Retrievers based on hybrid models have been shown to outperform both BM25 and neural models alone. Our approach exploits this improved performance when training a reranker, leading to a robust reranking model. The reranker, a cross-attention neural model, is shown to be robust to different first-stage retrieval systems, achieving better performance than rerankers simply trained upon the first-stage retrievers in the multi-stage systems. We present evaluations on a supervised passage retrieval task using MS MARCO and zero-shot retrieval tasks using BEIR. The empirical results show strong performance on both evaluations.
translated by 谷歌翻译
关于信息检索的许多最新研究集中在如何从一项任务（通常具有丰富的监督数据）转移到有限的其他各种任务，并隐含地假设可以从一个任务概括到所有其余的任务。但是，这忽略了这样一个事实，即有许多多样化和独特的检索任务，每个任务都针对不同的搜索意图，查询和搜索域。在本文中，我们建议使用几乎没有散热的检索，每个任务都有一个简短的描述和一些示例。为了扩大一些示例的功能，我们提出了针对检索器（即将到来）的及时基本查询生成，该查询将大型语言模型（LLM）作为几个弹片查询生成器，并根据生成的数据创建特定于任务的检索器。通过LLM的概括能力提供动力，即要来源使得可以仅基于一些示例{没有自然问题或MS MARCO来训练％问题生成器或双重编码器，就可以仅基于一些示例{没有}来创建特定于任务的端到端检索。出乎意料的是，LLM提示不超过8个示例，允许双重编码器在MARCO（例如Colbert V2）上训练的大量工程模型平均在11个检索套件中超过1.2 NDCG。使用相同生成数据的进一步培训标准尺寸的重新级别可获得5.0点NDCG的改进。我们的研究确定，查询产生比以前观察到的更有效，尤其是在给出少量特定于任务知识的情况下。
translated by 谷歌翻译
我们认为当前的红外标准，用于优化用户体验，测量太窄的IR空间的一部分。如果IR系统较弱，这些指标缺乏或完全过滤出需要改进的更深层次的文件。如果IR系统相对强，则这些指标欠更深的相关文档，这些文档可以在用户可消化的层次结构或文本摘要中呈现出甚至更强大的IR系统，这些文件甚至可以呈现来自数十或数百个相关文档的内容。我们从过去28年重新分析了超过70个TREC曲目，显示大约一半的欠压排名的文件，几乎所有的缺乏尾部文件。我们展示在2020年的深度学习轨道中，神经系统在排名第一的文件中实际上是近乎最佳的，而在尾部文件上只有BM25的适度增益相比。我们的分析基于简单的新系统导向度量，“雾化搜索长度”，它能够在任何深度准确且均匀地测量所有相关文档。
translated by 谷歌翻译
已经表明，在一个域上训练的双编码器经常概括到其他域以获取检索任务。一种广泛的信念是，一个双编码器的瓶颈层，其中最终得分仅仅是查询向量和通道向量之间的点产品，它过于局限，使得双编码器是用于域外概括的有效检索模型。在本文中，我们通过缩放双编码器模型的大小{\ em同时保持固定的瓶颈嵌入尺寸固定的瓶颈的大小来挑战这一信念。令人惊讶的是，令人惊讶的是，缩放模型尺寸会对各种缩放提高检索任务，特别是对于域外泛化。实验结果表明，我们的双编码器，\ textbf {g} enovalizable \ textbf {t} eTrievers（gtr），优先级％colbert〜\ cite {khattab2020colbertt}和现有的稀疏和密集的索取Beir DataSet〜\ Cite {Thakur2021Beir}显着显着。最令人惊讶的是，我们的消融研究发现，GTR是非常数据的高效，因为它只需要10 \％MARCO监督数据，以实现最佳域的性能。所有GTR模型都在https://tfhub.dev/google/collections/gtr/1发布。
translated by 谷歌翻译
我们提供了从文本到文本变换器（T5）的第一次探索句子嵌入式。句子嵌入式广泛适用于语言处理任务。虽然T5在作为序列到序列映射问题的语言任务上实现令人印象深刻的性能，但目前尚不清楚如何从编码器解码器模型生成陈列嵌入的句子。我们调查三种方法提取T5句子嵌入方法：两个仅利用T5编码器，一个使用全T5编码器解码器模型。为了支持我们的调查，我们建立了一个新的句子代表转移基准，SentGlue，它将Senteval Toolkit扩展到粘合基准的九个任务。我们的编码器的型号优于Senteval和SentGlue传输任务的句子 - BERT和SIMCSE句子嵌入，包括语义文本相似性（STS）。发现从数百万到数十亿参数的缩放T5产生一致的进一步改进。最后，我们的编码器 - 解码器方法在使用句子嵌入时在STS上实现了新的最先进的。我们的模型在https://tfhub.dev/google/collections/sentence-t5/1发布。
translated by 谷歌翻译
Retrieval-augmented in-context learning has emerged as a powerful approach for addressing knowledge-intensive tasks using frozen language models (LM) and retrieval models (RM). Existing work has combined these in simple "retrieve-then-read" pipelines in which the RM retrieves passages that are inserted into the LM prompt. To begin to fully realize the potential of frozen LMs and RMs, we propose Demonstrate-Search-Predict (DSP), a framework that relies on passing natural language texts in sophisticated pipelines between an LM and an RM. DSP can express high-level programs that bootstrap pipeline-aware demonstrations, search for relevant passages, and generate grounded predictions, systematically breaking down problems into small transformations that the LM and RM can handle more reliably. We have written novel DSP programs for answering questions in open-domain, multi-hop, and conversational settings, establishing in early evaluations new state-of-the-art in-context learning results and delivering 37-200%, 8-40%, and 80-290% relative gains against vanilla LMs, a standard retrieve-then-read pipeline, and a contemporaneous self-ask pipeline, respectively.
translated by 谷歌翻译
Scholarly text is often laden with jargon, or specialized language that divides disciplines. We extend past work that characterizes science at the level of word types, by using BERT-based word sense induction to find additional words that are widespread but overloaded with different uses across fields. We define scholarly jargon as discipline-specific word types and senses, and estimate its prevalence across hundreds of fields using interpretable, information-theoretic metrics. We demonstrate the utility of our approach for science of science and computational sociolinguistics by highlighting two key social implications. First, we measure audience design, and find that most fields reduce jargon when publishing in general-purpose journals, but some do so more than others. Second, though jargon has varying correlation with articles' citation rates within fields, it nearly always impedes interdisciplinary impact. Broadly, our measurements can inform ways in which language could be revised to serve as a bridge rather than a barrier in science.
translated by 谷歌翻译
We present NusaCrowd, a collaborative initiative to collect and unite existing resources for Indonesian languages, including opening access to previously non-public resources. Through this initiative, we have has brought together 137 datasets and 117 standardized data loaders. The quality of the datasets has been assessed manually and automatically, and their effectiveness has been demonstrated in multiple experiments. NusaCrowd's data collection enables the creation of the first zero-shot benchmarks for natural language understanding and generation in Indonesian and its local languages. Furthermore, NusaCrowd brings the creation of the first multilingual automatic speech recognition benchmark in Indonesian and its local languages. Our work is intended to help advance natural language processing research in under-represented languages.
translated by 谷歌翻译
The recent advent of large language models - large neural networks trained on a simple predictive objective over a massive corpus of natural language - has reinvigorated debate over whether human cognitive capacities might emerge in such generic models given sufficient training data. Of particular interest is the ability of these models to reason about novel problems zero-shot, without any direct training on those problems. In human cognition, this capacity is closely tied to an ability to reason by analogy. Here, we performed a direct comparison between human reasoners and a large language model (GPT-3) on a range of analogical tasks, including a novel text-based matrix reasoning task closely modeled on Raven's Progressive Matrices. We found that GPT-3 displayed a surprisingly strong capacity for abstract pattern induction, matching or even surpassing human capabilities in most settings. Our results indicate that large language models such as GPT-3 have acquired an emergent ability to find zero-shot solutions to a broad range of analogy problems.
translated by 谷歌翻译
This paper proposes a perception and path planning pipeline for autonomous racing in an unknown bounded course. The pipeline was initially created for the 2021 evGrandPrix autonomous division and was further improved for the 2022 event, both of which resulting in first place finishes. Using a simple LiDAR-based perception pipeline feeding into an occupancy grid based expansion algorithm, we determine a goal point to drive. This pipeline successfully achieved reliable and consistent laps in addition with occupancy grid algorithm to know the ways around a cone-defined track with an averaging speeds of 6.85 m/s over a distance 434.2 meters for a total lap time of 63.4 seconds.
translated by 谷歌翻译